A theoretical comparison of batch-mode, on-line, cyclic, and almost-cyclic learning

نویسندگان

  • Tom Heskes
  • Wim Wiegerinck
چکیده

We study and compare different neural network learning strategies: batch-mode learning, online learning, cyclic learning, and almost-cyclic learning. Incremental learning strategies require less storage capacity than batch-mode learning. However, due to the arbitrariness in the presentation order of the training patterns, incremental learning is a stochastic process; whereas batch-mode learning is deterministic. In zeroth order, i.e., as the learning parameter eta tends to zero, all learning strategies approximate the same ordinary differential equation for convenience referred to as the "ideal behavior". Using stochastic methods valid for small learning parameters eta, we derive differential equations describing the evolution of the lowest-order deviations from this ideal behavior. We compute how the asymptotic misadjustment, measuring the average asymptotic distance from a stable fixed point of the ideal behavior, scales as a function of the learning parameter and the number of training patterns. Knowing the asymptotic misadjustment, we calculate the typical number of learning steps necessary to generate a weight within order epsilon of this fixed point, both with fixed and time-dependent learning parameters. We conclude that almost-cyclic learning (learning with random cycles) is a better alternative for batch-mode learning than cyclic learning (learning with a fixed cycle).

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Antioxidant Properties of Ajwain using Square wave, Cyclic voltammetry methods and DPPH method

Ajwain is one of the medicinal plants, which the highest composition is thymol, as a strong antioxidant respect to the obtained chromatograms of GC/MS. The antioxidant activity of the Ajwain is measured by square wave voltammetry method and cyclic voltammetry method and 2, 2-Diphenyl-1-picrylhydrazyl (DPPH) method in at the specific concentrations of 1%, 1.5%, 2% and 2.5% and constant pH. The r...

متن کامل

Deterministic convergence of conjugate gradient method for feedforward neural networks

Conjugate gradient methods have many advantages in real numerical experiments, such as fast convergence and low memory requirements. This paper considers a class of conjugate gradient learning methods for backpropagation (BP) neural networks with three layers. We propose a new learning algorithm for almost cyclic BP neural networks based on PRP conjugate gradient method. We then establish the d...

متن کامل

Theoretical investigation on the aromaticity of mono-substituted benzene derivatives by using cyclic reference

The degree of aromaticity of mono-substituted derivatives of benzene has beeninvestigated using a new index based on electric field gradient index, by using two mechanicalquantum methods with Gaussian 03. Two different basis sets have applied to study and theresults compared. This strategy has demonstrated that, due to violation of symmetry in have pisystems,how the degree of aromaticity can ha...

متن کامل

Selective Binding of Cyclic Nanopeptide with Halides and Ion Pairs; a DFT-D3 Study

In this article, theoretical studies on the selective complexation of the halide ions (F¯, Cl¯ and Br¯) and ion pairs (Na+F¯, Na+Cl¯ and Na+Br¯) with the cyclic nano-hexapeptide (CP) composed of L-proline have been performed in the gas phase. In order to calculate the dispersion interaction energies of the CP and ions, DFT-D3 calculations at the M05-2X-D3/6-31G(d) level was employed. Based on t...

متن کامل

Theoretical study of the effect of internal strain on the bond length and rate of hydrolysis in cyclic amides

The internal strain in cyclic amides are explained as a factor of resonance that are effected on the bond length C-N  and are a major factor of rates of hydrolysis. The cyclic amides in this study are optimized by Gaussian program and the bond length of C-N in the rings are studied by HF/6-31G*.

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • IEEE transactions on neural networks

دوره 7 4  شماره 

صفحات  -

تاریخ انتشار 1996